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ABSTRACT 


Troubleshooting in tactical data networks is often performed with a common toolset of 
programs, such as ping, traceroute, and protocols such as Simple Network Management 
Protocol. The assumption with such tools and protocols is that the logical configuration 
of the network is correct; if it is not, these tools could fail or return inconclusive results. 
While failure can be useful to prove a problem exists, it often does not provide enough 
data to actually diagnose the issue. Protocols such as Link Layer Discovery Protocol ex¬ 
ist to troubleshoot from the data-link layer, but these protocols cannot operate between 
subnets. This limits their usefulness in tactical networks. An active networking project 
known as XPLANE has been developed at the Naval Postgraduate School with these issues 
in mind. XPLANE allows network operators to take active measurements in a network 
without relying on the logical layer. This ability is extremely important in live tactical 
networks, particularly when there is significant geographic separation between nodes. Be¬ 
fore XPLANE can be used in tactical networks, important issues around security and the 
XPLANE’s user interface must be resolved. This thesis explores the relevance of XPLANE 
in tactical networks and develops a front-end to XPLANE for tactical network operators. 
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Executive Summary 


The author of this thesis deployed to Operation Enduring Freedom-Afghanistan in 2011- 
2012 as the communications officer for 2nd Battalion 11th Marines (2/11). During his time 
in Afghanistan, the author gained first-hand experience of managing and troubleshooting 
a network geographically dispersed throughout a combat zone. Configuration issues at 
remote sites affected network performance, which had impacts on 2/1 l’s ability to do its 
mission. The majority of network issues were due to human error. These issues often 
rendered the Internet Protocol (IP) layer of the network, often called the logical layer, 
unreliable. The author quickly learned that common network troubleshooting tools such as 
ping, traceroute, and Simple Network Management Protocol have limited utility in logically 
broken networks. Basic troubleshooting became impossible, and Marines were often sent 
to remote sites to physically troubleshoot the network, which wasted valuable time and 
manpower. 

The common suite of troubleshooting tools and protocols all suffer from the same weak¬ 
ness. They rely on the very thing that is often the source of the problem: the network’s 
logical configuration. When a network has logical routing loops, unreliable transmission 
paths, and other anomalies, traditional tools become unreliable. Protocols such as Link 
Layer Discovery Protocol that operate from the data-link layer are bounded to their local 
subnet; they are unable to move between networks and are of limited use. An active net¬ 
working system known as XPLANE has been developed at the Naval Postgraduate School 
with these issues in mind. 

XPLANE utilizes the concept of active networking. The goal of the active networking 
paradigm is to increase capabilities in traditional networks by making them programmable. 
XPLANE programs run inside frames in the network. This allows XPLANE programs to 
discover and traverse network architecture to diagnose malfunctioning segments. XPLANE 
lives at the data-link layer of the network, which allows it to function in networks that are 
logically broken. It has unique capabilities that make it well suited for tactical networks. 
First, XPLANE programs can relocate throughout the network via the data-link layer. Sec¬ 
ond, programs are able to inject and capture packets in the network. When combined, 
these capabilities allow for active measurements to be taken from anywhere in the net- 




work that has data-link layer connectivity. This feature could have a profound impact on 
troubleshooting in geographically dispersed tactical networks. 

The problem with XPLANE is one of accessibility to the common network operator. In its 
current state, XPLANE consists of the runtime environment and a functional programming 
language to create XPLANE applications. There is no user interface or mechanism for 
tracking the results of multiple XPLANE programs. This thesis addresses these issues and 
creates a practical troubleshooting toolset for tactical networks. First, a comprehensive set 
of XPLANE applications known as the tactical edge suite has been developed. This suite 
of applications can map network topology from scratch and diagnose common network 
anomalies. Second, an XPLANE serx’er has been developed that is responsible for launch¬ 
ing XPLANE applications on the network and interpreting their results. The XPLANE 
server additionally keeps state of the network as discovered by the tactical edge suite. Fi¬ 
nally, a web-based user interface (UI) for the XPLANE server has been developed. The 
web-based UI gives network operators a visual representation of the network and highlights 
any detected anomalies. 

To test the XPLANE-based toolset, a small test network was created. This test network con¬ 
sisted of four routers running the XPLANE software and six client computers which did not 
run XPLANE. A number of anomalies were introduced into the network: duplicate and in¬ 
correct IP addresses were used, subnet addresses were entered incorrectly into routers, and 
logical routing loops were introduced. Network discovery and follow-on troubleshooting 
was first attempted with common troubleshooting tools discussed earlier. Attempts were 
then made to diagnose the same problems using the XPLANE-based toolset, and the results 
from both were compared. The XPLANE-based toolset outperformed traditional tools for 
both network discovery and identifying anomalies. 

While this thesis addresses some of the issues that need attention before XPLANE becomes 
a practical tool, there remain ways it can still be improved. Some of them are discussed. 
They include security, improved packet injection, a more powerful user interface and a 
richer tactical edge suite of applications. 
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CHAPTER 1: 
Introduction 


In late 2011, the author of this thesis deployed to Operation Enduring Freedom (OEF) as 
the communications officer for 2nd Battalion 11th Marines (2/11). As the only artillery 
battalion in Helmand Province, 2/1 l’s mission was to provide supporting fires to any unit 
in the area of operations, which stretched from the Kajaki Dam to the Helmand/Pakistan 
border. In order to cover such a large area, 2/11 was segmented into small, platoon-sized el¬ 
ements that were geographically dispersed throughout Helmand Province to provide max¬ 
imum coverage. As a consequence of this dispersion, 2/11 became extremely reliant on 
Helmand’s Secret Internet Protocol Router Network (SIPRnet) in order to receive and pro¬ 
cess calls for fire. Most units were well out of radio range from the artillery position that 
supported them, and the protocols to have fire missions approved by higher headquarters 
often required the use of SIPRnet for coordination. 

By the midpoint of his deployment, the author had hit a wall. Nearly every artillery po¬ 
sition in Helmand Province was having serious SIPRnet connectivity issues, which was 
causing delays in fire missions. Delays in fire missions meant that Marines in a firefight 
on the ground would have to sit there and wait for the support they requested, which was 
dangerous and unacceptable. The problem became so bad that meetings were occurring on 
a day-to-day basis between the 2/11 Communications section and the Division G-6, which 
was the agency responsible for the entire Helmand network. The conclusion of most trou¬ 
bleshooting sessions was the same; the problem was often too difficult to diagnose and fix 
from the central hub of the network, Camp Leatherneck, and someone would have to travel 
to different forward operating bases (FOBs) and physically troubleshoot the issue. By the 
end of the deployment, the author and his Marines had travelled to five different FOBs 
throughout Helmand Province to physically troubleshoot issues. These issues almost al¬ 
ways ended up being caused by human error. Other units to include the Division G-6 and 
the regimental combat team (RCT) 8 and RCT 6 Communications Section also travelled to 
artillery firing positions to troubleshoot the network throughout 2/1 l’s deployment. 

Sending Marines to fix network issues was not a trivial decision. The threat of improvised 
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explosive devices (IEDs) prevented most personnel from travelling via a ground convoy, 
especially for long distances. This made aircraft the primary mode of personnel transporta¬ 
tion across Afghanistan. Unfortunately, travelling by aircraft was also risky, and at best 
would take two to four days for a round trip flight, depending on weather. All of these 
issues caused the author to re-evaluate the situation. How did it get this bad? How did 
sending Marines become the fix to every network issue we experience? The answer was 
simple: the network was too dispersed and too complex for the tools available at the time. 
The Helmand communications network had become a mesh of interconnected FOBs span¬ 
ning across hundreds of kilometers. The redundant architecture, while tactically sound, 
made equipment configurations nearly impossible to troubleshoot from a central location. 
Simple tools like ping and traceroute were unable to help, mostly due to the fact that such 
tools rely on the very thing that needs to be fixed: the network configuration. More com¬ 
plex tools such as Solar Winds were in use, but unfortunately could only offer a picture of 
what they could see from the central hub of the network, Camp Leatherneck, which added 
no value. The last bastion of hope, the documentation for the entire Helmand Province net¬ 
work, was only accurate up to late 2011. What the Marines of Helmand Province needed 
were tools that would allow them to ask deeper questions about the network. They needed 
tools that could utilize the network segments that were working correctly, such as the link- 
layer of the various transmissions systems, to gain insight into the malfunctioning upper 
layers. Such tools could have prevented the need to send additional Marines into harm’s 
way, simply to troubleshoot network segments that were physically out of reach. 

1.1 Overview and Problem Statement 

Network troubleshooting is often performed with a suite of commonly available tools such 
as ping, traceroute, and Simple Network Management Protocol (SNMP). These tools can 
be used to perform common tests such as connectivity between nodes, path discovery be¬ 
tween nodes, and in the case of SNMP, statistics and detailed information on node con¬ 
figurations can be queried from a distance. The assumption with all such tools is that the 
logical configuration of the network is correct. If it is not, tests performed with these tools 
will likely fail. While failure can be useful to prove a problem exists, it often does not 
provide enough data to figure out why the problem exists in the first place. 

The common weakness in these tools lies at the heart of the original assumption made: 
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how can tools that rely on the logical layer diagnose issues with the logical layer itself? 
The obvious answer is they often cannot. In order to troubleshoot issues in the logical 
layer, one has to move “back down” the layers of the network. In the case of the open 
systems interconnection (OSI) model, this means having tools that work at layer two, often 
called the data link layer. At the data link layer, there are a number of protocols that 
aid in network discovery such as the Address Resolution Protocol (ARP) [1], Link Layer 
Discovery Protocol (LLDP) [2], and Microsoft’s Link Layer Topology Discovery (LLTD) 

[3] . There are also tools such as linkloop which replicate the behavior of the ping utility 

[4] . Unfortunately, all of the aforementioned tools and protocols only aid in one aspect 
of network troubleshooting: discovery. Additionally, such tests are limited to the local 
broadcast domain, which means a node can only discover what it is directly connected 
to. This is a limitation of operating at the data link layer; the network is organized into 
non-routable segments that are unable to communicate without relying on layer three, the 
logical layer. 

1.2 Research Questions 

The need for tools that operate at the data link layer, yet are able to traverse networks 
beyond the local broadcast domain is apparent. Such tools could take advantage of physical 
connections that still exist between nodes to troubleshoot logical layer issues. What would 
the requirements of such a tool be? With no routing information inherently available at 
the data link layer, packets would have to somehow be able to make decisions about their 
path as they traverse the network. In other words, packets would need to be active instead 
of a collection of passive data. This concept, known as active networking, is not a new 
one [5]. The software defined networking (SDN) paradigm, which has gained popularity 
in recent years, has its roots in the idea of programmable networks [6]. By extending 
the functionality of traditional networks through active networking, new systems can be 
developed to manage, troubleshoot, and even reconfigure a running network in ways not 
possible today. 

One such system, known as XPLANE, is currently in development at the Naval Postgrad¬ 
uate School (NPS) [7]. Once enabled on a network, XPLANE allows for small programs 
to be run on the network itself. XPLANE programs have two key capabilities: the abil¬ 
ity to crawl through the network via the data link layer, as well as the ability to inject 
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and capture packets in the network [7]. Used together, these capabilities allow for active 
measurements to be performed from any point in the network that has link-layer connec¬ 
tivity. When considered in the context of tactical data networks, capabilities such as these 
have the potential to change the way network troubleshooting is performed. This thesis 
will explore the following question: how can active networking systems such as XPLANE 
improve troubleshooting in malfunctioning or damaged tactical data networks? 

The XPLANE has been demonstrated in laboratory settings, however work remains to make 
it a practical tool for troubleshooting tactical networks. First, a set of XPLANE applica¬ 
tions that would aid in troubleshooting common issues in tactical networks needs to be 
identified and developed. Second, an interface needs to be developed for launching these 
applications and capturing the results. XPLANE applications run asynchronously on the 
network; results of any measurements taken are returned back to the node that initiated the 
program and then forgotten. It is up to the user to keep state and interpret results in a mean¬ 
ingful way. In order to provide accessibility to the XPLANE results, a visual user interface 
needs to be developed. Finally, introducing new software such as XPLANE into a network 
presents security risks. What are the security risks, and how will they be mitigated? 

To address these issues, a comprehensive set of XPLANE applications known as the tacti¬ 
cal edge suite has been developed to map the topology of a network and diagnose common 
network anomalies. These applications are managed and launched via a XPLANE serx’er 
that is responsible for capturing and interpreting results of programs ran on the network. 
The XPLANE server is also responsible for keeping state of the network, which can be 
visualized via a web-based user interface (UI) to the server. The XPLANE web-based 
interface displays the state of the network as a graph of interconnected nodes. The inter¬ 
face will also highlight any anomalies detected in the network. Security concerns with 
XPLANE have been identified, and access control mechanisms have been incorporated to 
ensure that only authorized users are allowed to execute code on the network. Additionally, 
the topic of key management has been explored and a model for access control has been 
developed. 
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1.3 Scope of Research 

The research conducted in this thesis will be limited to detection and diagnosis of issues 
in tactical data networks; no attempt will be made to remotely fix issues diagnosed with 
XPLANE. Changing the configuration of live network devices presents its own set of chal¬ 
lenges that include dealing with multiple proprietary interfaces and access control mecha¬ 
nisms [6]. Additionally, XPLANE is limited to observing network behavior by design and 
therefore is not capable of modifying network behavior. Possible extensions to this thesis 
and the results provided will be discussed in Chapter 7 under future work. 

1.4 Thesis Organization 

Chapter 2 begins with an overview of the OSI model and basic networking concepts. Com¬ 
mon issues in tactical data networks are presented, and the traditional tools and protocols 
used in network troubleshooting are explained, with emphasis on their shortfalls in tacti¬ 
cal environments. Recent advances in network management systems is presented, and the 
concept of active networking is explained. XPLANE, an active networking system, is intro¬ 
duced. Active networking research projects that inspired XPLANE’s design are addressed, 
and their limitations in tactical environments are discussed. 

Chapter 3 presents XPLANE and XPLANE Programming Language (XPL). An overview 
of XPLANE and XPL’s capabilities and limitations is given. The mechanics of XPLANE 
at the data-link layer of the network is explained. Programming in XPL is addressed, and 
code examples are given to demonstrate XPL constructs to the reader. 

Chapter 4 presents the tactical edge suite for troubleshooting tactical networks. The design 
philosophy for the suite is presented, along with the testing methodology and platforms 
used. Lor each application developed, the rationale behind developing it is explained, as 
well as a general description of the algorithm, any assumptions made, and additional con¬ 
cerns. 

Chapter 5 presents the design of an XPLANE server, and the design of the web-based UI. 
The composition of the server and its integration with the tactical edge suite is presented. 
The web-based UI and its interface to the XPLANE server is described. 

Chapter 6 presents the results of an assessment made of the utility of the XPLANE server 
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and the tactical edge suite. The XPLANE server and tactical edge suite are compared with 
traditional network tools for troubleshooting networks. The results of this comparison are 
given. 

Chapter 7 provides some conclusions and addresses future work of both XPLANE itself as 
well as the work presented in this thesis. 

Appendix A contains all code listings referenced in this thesis. 

Appendix B contains the detailed configuration for the network used in Chapter 6. 
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CHAPTER 2: 

Background and Related Work 


Computer networks have greatly evolved over the past few decades. The amount of devices 
on networks, as well as the functionality of each device, has increased over time [6]. Tradi¬ 
tionally, networks have been configured "by hand," (i.e., manually configured). As of this 
writing, this is the standard method of network configuration in the United States Marine 
Corps (USMC). Routers, switches, firewalls, and combination devices must be configured 
in a logical manner that allows for proper operation of all devices on the network [6]. As the 
amount of devices on a network increases, manually managing all devices and their con¬ 
figurations from a central point becomes difficult. In the case of geographically-dispersed 
networks in an active combat zone, the idea is untenable. In order to address the issues 
of managing complex networks, one has to understand why traditional tools and protocols 
have hit the limits of their capabilities, as well as previous and current work related to these 
issues. 


2.1 OSI Model 

The OSI model is a concept used to describe the various layers of computer networks and 
how the layers relate to each other [1], The model breaks networks down in to seven distinct 
layers: physical, data link, network, transport, session, presentation, and application [1], 
Layers are typically numbered from one to seven starting from the physical layer, and each 
layer is defined by the function it performs in the network [8]. This thesis will focus on the 
data link layer (layer two) and the network layer (layer three). 

2.1.1 Data Link Layer 

The function of the data link layer is to provide error-free transmission between adjacent 
nodes over the physical layer [8]. Individual data packets are referred to as frames at this 
layer [1], Many different link layer protocols exist; common examples are Point-to-Point 
Protocol (PPP), Fiber Distributed Data Interface (FDDI), and the Institute of Electrical and 
Electronics Engineers (IEEE) 802.3 standard, more commonly referred to as Ethernet. Eth¬ 
ernet is the most common link layer protocol in use today [1], Asa service, Ethernet offers 
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flow control, error detection, and error correction on data transmitted between adjacent 
nodes [1], This thesis will focus on using Ethernet as the link layer protocol. 

2.1.2 Network Layer 

Transmission of data at the link layer is limited to adjacent nodes. In order to route packets 
between networks, the link layer forwards the contents of frames to the network layer. This 
layer is also commonly referred to as the logical layer. The function of the network layer is 
to provide end-to-end delivery of data between nodes of different networks [1]. The most 
common network layer protocol is the IP. IP provides end-to-end delivery of datagrams , 
i.e. packets, via best effort delivery [1]. This thesis will focus on using IP (version 4) as 
the network layer protocol. 

2.2 Network Issues that Require Troubleshooting 

The logical layer of the network, in particular IP, is necessary to create larger networks 
from subnetworks [1]. The ability to route packets from one network to another is what 
makes networks such as the Internet possible. In order to route packets efficiently, IP 
relies on routing algorithms to calculate paths on its behalf [1]. Routing algorithms in turn 
rely on a set of assumptions about the network and how it is configured. One important 
assumption is that each node in the network has a unique IP address , which identifies it 
within the network. Another assumption is that the routers themselves have been correctly 
configured. When these assumptions are proved false, errors will begin to occur in the 
network. This thesis will refer to network errors as anomalies. 

2.2.1 IP Address Anomalies 

IP addresses are used to uniquely identify nodes within a network. When two or more nodes 
use the same IP address, this is referred to as duplicate IP addresses within the network. 
Duplicate IP addresses present a problem to network devices; how will a decision be made 
on where to forward data when there are multiple nodes using the same IP address? The 
behavior of network devices in this situation will depend on many factors, such as ARP 
table caches. 

Along with IP addresses comes the notion of subnets and subnet masks, which are used 
to segment networks into distinct sub-networks [1]. Subnet masks have to be correctly 
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configured on nodes and routers, or else routing algorithms will be unable to make a deter¬ 
mination on which subnetwork a given IP address lives. Subnet masks can be accidentally 
misconfigured on routers, end hosts, or both. The effect on the network will depend on 
many factors, making misconfigured subnet masks difficult to remotely diagnose and trou¬ 
bleshoot. 

2.2.2 Packet Loss 

IP is a best effort service, and therefore makes no guarantees that packets will be delivered 
to their destination [1], For reliable end-to-end transmission, protocols such as the Trans¬ 
mission Control Protocol (TCP) are implemented at layer four of the network stack, the 
transport layer [1], Packet loss can occur in a network for a variety of reasons; examples 
include transmission errors, routing loops, firewalls, access control list (ACL) restrictions, 
and queueing delays in routers. While it is easy to detect packet loss, it is often hard to 
single out the source of the error. 

2.2.3 Routing Loops 

Routing loops occur in networks when packets are unable to leave an infinitely looping path 
in the network [9]. Figure 2.1 shows an example of a routing loop that has been formed 
between three routers in a network. Routing loops can occur for many different reasons; 
examples include incorrectly configured static routes, delays in network convergence, and 
general router misconfiguration [9]. As a network anomaly, routing loops have unique side 
effects depending on the network topology and configuration, and are therefore hard to 
detect in many networks [9]. 

2.3 Traditional Tools for Troubleshooting 

Over time, network administrators have come to rely on a common suite of tools and pro¬ 
tocols to troubleshoot networks. These tools are usually available on all end-user operating 
systems as well as network devices such as routers. This section will discuss the more 
common tools and protocols to address the shortfalls of each one. This section is not a 
comprehensive list of every troubleshooting tool in existence; there are simply too many. 
Most if not all are similar enough in function to the tools and protocols that are addressed, 
and suffer from the same weaknesses. 
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Figure 2.1: An example of a routing loop between three nodes. 


2.3.1 Ping 

The ping utility is a common troubleshooting tool that uses the Internet Control Message 
Protocol (ICMP) to test end-to-end connectivity between nodes. It typically comes in the 
form of a command-line tool, however the basic ICMP echo request/response mechanism 
is often referred to as "pinging a host", regardless of the way the message was sent. A 
successful ping implies end-to-end connectivity between two nodes on a network. A failed 
ping does not necessarily imply a lack of connectivity between two nodes; it simply in¬ 
dicates there could be an issue. Host-based firewalls, network firewalls, ACL restrictions, 
and other factors could all cause ping to fail, even if there is a logical path between the two 
nodes in question. This limits ping’s utility in troubleshooting environments. 

2.3.2 Traceroute 

The traceroute utility is another command-line troubleshooting tool. Traceroute makes use 
of the time to live (TTL) field of IP datagrams to trace the route taken to a given destination 
[1]. A traceroute analysis of the path between two nodes will yield a list of hops along 
the path, the round-trip delay to each hop, and the hostnames and/or IP addresses of each 
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node along the path [1]. Just like ping, traceroute utilizes ICMP to send messages. There 
are versions that can alternatively use TCP for networks that restrict ICMP messages. The 
information provided by traceroute can be very useful in certain troubleshooting situations. 
When the logical layer of the network is malfunctioning, the utility of traceroute can vary 
widely. 

2.3.3 SNMP 

SNMP is a network management protocol used to manage devices and query information 
from them over a network [1], SNMP organizes information into objects known as manage¬ 
ment information bases (MIBs). MIBs are used to organize information in a hierarchical 
fashion with a well-defined structure that is standard across all devices. In turn, MIBs can 
be viewed as a database of configuration parameters and statistics that can be reached from 
other nodes in the network [1], There are standard MIBs for information such as device 
configuration, protocol statistics, and routing tables and configurations. 

In comparison to command-line utilities such as ping and traceroute, SNMP is an actual 
management protocol with much more capability. SNMP has far more utility in certain 
troubleshooting situations than ping and traceroute, but unfortunately suffers from the same 
weaknesses. SNMP is a network layer (layer three) protocol, and its capability in a mal¬ 
functioning network varies widely. 

2.3.4 LLDP, LLTD, and CDP 

The IEEE LLDP, Microsoft’s LLTD, and Cisco’s Cisco Discovery Protocol (CDP) are link 
layer protocols used by nodes to advertise information to the rest of the local area network 
[2,3,10]. This information can include information such as a node’s identity, neighbors, 
and services available [2, 3,10]. The three protocols are similar enough in functionality 
to address their capabilities and limitations as a group. While LLDP, CDP, and LLTD are 
useful as link layer discovery tools, they are by definition limited to the local broadcast 
domain and therefore unable to help in discovering nodes across network boundaries. 

2.4 Recent Advances in Network Troubleshooting 

In addition to the tools and protocols mentioned, there are a number of successful academic 
projects that have created new and more powerful network management and troubleshoot- 
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ing capabilities. 


2.4.1 Sophia 

Sophia is a project aimed at developing a network information plane to collect, store, prop¬ 
agate, and react to observations about the network in a distributed manner [11]. Sophia 
is a good example of the overlay network concept, which is a networked system of nodes 
within the larger network. Sophia creates an overlay network of distributed sensors that 
provide information to a declarative programming environment that evaluates logic state¬ 
ments about the network [11]. Sophia is novel in the fact that administrators can query the 
network directly for management information in a declarative manner [11]. In other words, 
administrators can ask deep questions about network state, without telling the network how 
to find the answer to the query. While Sophia shares similar goals with XPLANE, it relies 
on the assumption that the logical layer of the network is functioning correctly, limiting its 
usefulness in malfunctioning networks. 

2.4.2 NetQuery 

NetQuery is a project from Cornell University aimed at implementing a knowledge plane on 
large-scale networks such as the Internet [12]. Similar to Sophia, NetQuery aims to dissem¬ 
inate information about network entities to support application-level reasoning. NetQuery 
focuses on trusted computing, information attribution, and how to reason about the network 
with limited trusted information [12]. NetQuery, like Sophia, makes assumptions on the 
correctness of the logical layer of the network, limiting its usefulness in malfunctioning 
networks. 

2.5 Active Networking 

The term active networking refers to a new networking paradigm developed in the 1990s 
[5]. The goal of this paradigm was to generate innovation in the design and control of net¬ 
works by making them programmable [6]. By making networks programmable, followers 
of the active networking paradigm were looking for novel ways to create new capabilities 
and services. Traditional computer networks are not programmable in the classic sense; 
devices are individually configured, and there is no environment on the network for code 
to be executed within [6]. Active networking changes this by presenting two candidate net¬ 
work programming models; this thesis will focus on the capsule model [6]. In the capsule 
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model, packets are a combination of code and data known as capsules [5]. The code in these 
capsules can be executed at any node in the network and can modify network behavior [5]. 

This radical approach to opening up network capabilities may have been ahead of its time; 
active networking never experienced widespread adoption beyond the research commu¬ 
nity [6]. A possible reason for this was the lack of a clear demand for such capabilities at 
the time [6]. This is no longer the case. In many ways, the active networking community 
was attempting to address issues now being addressed by SDN [6]. While there are simi¬ 
larities between SDN and active networking, SDN is primarily concerned with the idea of 
separating the control plane of a network from the data plane in order to dynamically adjust 
the control plane based on network conditions [6]. In other words, SDN intends to make 
networks more dynamic by standardizing the control of network devices such as routers 
and switches from a central location. Active networking in the capsule model focuses on 
manipulation from the data plane instead; this means that it is the responsibility of the cap¬ 
sules, not the routers, to dynamically adjust to network conditions [6]. The importance of 
this distinction in tactical networks will be explained further in Chapter 3. SDN and the 
centralization of the control plane will likely have a role in future tactical networks, but it 
will not be a focus of this thesis. 

The active networking paradigm has manifested itself through a number of academic 
projects [6]. XPLANE is only one of them. In order to understand why XPLANE is a 
better fit for tactical networks than the others, we need to address prior and related work. 


2.5.1 PLAN 

Programming Language for Active Networks (PLAN) is an active networking project de¬ 
veloped at the University of Pennsylvania in the late 1990s. The goal of PLAN was to 
develop a functional programming interface to the concept of active networking [13]. The 
developers of PLAN envisioned using it as a network-level ’glue’ language for services 
written in other general-purpose languages [13]. Diagnostic programs such as ping and 
traceroute can be emulated in PLAN with minimal effort, and more importantly, without 
relying in ICMP. This kind of flexibility demonstrates the ability to replace the functional¬ 
ity of certain protocols with simple programming interfaces [13]. The design of XPLANE 
was partially inspired by PLAN, but the two platforms differ in an important aspect; PLAN 
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lives at the network layer, while XPLANE lives at the data link layer [7,13]. Addition¬ 
ally, PLAN is not focused on network troubleshooting, and lacks features such as packet 
injection to take active network measurements. The PLAN project is no longer maintained. 

2.5.2 Sprocket 

The Sprocket programming language is a component of the Smart Packets architecture, an 
active networking project focused on network monitoring and management [14]. Similar 
to PLAN and XPLANE, Smart Packets are active network programs that execute on nodes 
as they traverse the network [14]. Sprocket, a programming language based on the syntax 
of C, serves as the high-level language that Smart Packets are written in [14]. The de¬ 
sign of XPLANE and the XPL is influenced by Smart Packets and Sprocket [7]. As with 
PLAN, Smart Packets operates at the network layer and therefore has limited use in the 
types of networks this thesis intends to explore. Smart Packets is not focused on network 
troubleshooting, as is the case with PLAN, and similarly lacks active network measurement 
capability. 

The Active Networking research efforts discussed so far have a common weakness: they 
are either unable to function in logically broken networks, or were not specifically designed 
to troubleshoot networks. Both of these shortfalls limit the usefulness of such systems in 
tactical environments. An active networking system known as XPLANE has been devel¬ 
oped with these issues in mind. The next chapter will introduce XPLANE and give an 
overview, with code examples, of XPLANE’s capabilities. 
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CHAPTER 3: 
XPLANE 


XPLANE is an active networking system developed at NPS. The goal of XPLANE is to 
develop a system that can perform active measurements for troubleshooting in a network 
relying solely on physical connectivity [7]. In other words, XPLANE aims to function cor¬ 
rectly even if the networking is logically broken. This feature, along with programmable 
packet injection, make XPLANE suitable for troubleshooting and separate it from other 
active networking efforts. This thesis will focus on using XPLANE to diagnose anoma¬ 
lies in networks, and compare the utility of information provided by XPLANE against the 
common troubleshooting tools and protocols mentioned earlier. 

3.1 Overview 

XPLANE can be considered an overlay network that exists just above the data link layer [7]. 
The system consists of a shim that operates at the data link layer of network devices, and a 
programming language, XPL, that is used to create programs that will run on the XPLANE. 
The shim provides the runtime environment for XPLANE programs. When an XPLANE 
packet is received on an XPL-enabled node, the shim decodes the packet and executes the 
XPL code inside. Figure 3.1 shows the format of an XPLANE packet. The current version 
of XPLANE does not utilize the Sender ID, Sequence Number, or Fragmentation fields. It 
is important to note that the marshaled code inside the packet might be the continuation of 
an ongoing computation in the network. XPL programs can relocate themselves throughout 
the network during the execution of a program as they need [7]. 


Marker Version 

Packet Length 

Sender ID 

Receiver ID 

Sequence Number 

Fragmentation 

Checksum 

Authentication (20 bytes) 

Marshaled Code 


Figure 3.1: XPLANE packet format 
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XPLANE does not rely on the logical layer for transportation through a network. It instead 
relies on link layer broadcasts between nodes. This allows XPL programs to move through 
networks that are logically broken even when normal IP traffic cannot. If an executing 
program chooses to relocate, the XPLANE shim will generate an XPLANE packet with 
the execution’s current state stored in it, and broadcast the frame on the data link layer to 
all adjacent nodes. XPLANE uses the notion of XPLANE node IDs, which are unique 
node identifiers throughout the XPLANE. Nodes identify themselves as recipients of an 
XPLANE packet based on the node ID of the receiver. Packets can be addressed to specific 
node IDs. Packets can additionally be addressed to all adjacent nodes. XPLANE refers 
to this as flooding. XPLANE programs aim to fit in a single Ethernet frame, which has a 
maximum size of 1500 bytes. XPLANE will remember neighboring XPLANE nodes once 
they are discovered. The XPLANE shim can also be configured to beacon its presence 
on the local subnet to all other XPL-enabled nodes. Beaconing helps in discovering di¬ 
rectly connected neighbors, increasing the utility of XPL programs that rely on neighbor 
information. 


3.2 XPL 

XPL is a subset of the TinyScheme functional programming language. It has built-in con¬ 
structs to perform functions and access node information specifically related to XPLANE. 
Table 3.1 gives a summary of common XPL-Scheme functions, their pseudocode syntax 
for this thesis, and a description of functionality. The first feature of XPL to explore is the 
ability to relocate to a different node. This can be accomplished in two different ways with 
XPLANE. The first way is the On function, which has the syntax: 


On(el, e2) 


where el is an expression that evaluates to the node ID of a directly connected neighbor, 
and e2 is the expression to evaluate on that neighbor. The expression evaluated on the 
neighboring node can be any legal expression, including more calls of the On function: 

<9/?(7, <9/7(2, (On 3, (2 + 2)))) 
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This program would relocate to the XPLANE node with node 1, then relocate to node 2, 
then relocate to node 3, and finally evaluate the expression 2+2 on node 3. Figure 3.2 
shows how the computation would take place on a network assuming the nodes are directly 
connected. 


XPL-Scheme 

Pseudocode 

Description 

(on n e) 

On(n,e ) 

Relocate to node n and evaluate expression e 

(onflood e) 

OnFlood(e) 

Relocate to all directly-connected neighbors 
and evaluate expression e 

node 

Node() 

Returns node ID for current node 

(node, if aces) 

Node.ifacesi) 

Returns list of network interfaces for current 

node 

(node.ip i) 

Node.ip(i ) 

Returns IP address for interface i on current 

node 

(node.mask i) 

N ode .mask(i) 

Returns subnet mask for interface i on current 

node 

(node.ethaddr i) 

Node.ethaddr(i) 

Returns MAC address for interface i on current 

node 

(node.direct i) 

Node, direct (/) 

Returns list of known neighbors on interface i 
of current node 

(send el e2 e3) 

Send(el,e2\ e3) 

el evaluates to device to send on, e2 evaluates 
to list specifying protocol and destination, and 
e3 represents a body of code to be evaluated 

(pcap el e2 e3 e4) 

Pcap(el,e2,e3,e4) 

el evaluates to device to listen on, e2 evaluates 
to protocol, e3 evaluates to timeout value for lis¬ 
tener, e4 evaluates to packet processing function 

(list...) 

List {) 

Returns arguments to function as a list 

(car 1) 

Head(l) 

Returns the first element of the list 1 

(cdr 1) 

Tail(l) 

Returns all but the head of the list l 

(append 1...) 

AppendToList(l ,...) 

Appends all supplied arguments after list / to / 

(member s 1) 

M ember (s, /) 

Returns true if element s is a member of list /. 
Returns false otherwise 


Table 3.1: XPL-Scheme and Pseudocode syntax for XPLANE functions 

A more useful example of code relocation is to retrieve information about a node on the 
network: 


<9/?(7, <9/7(2, On(3, (Node.ip(ai))))) 
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(on 1 (on 2 (on 3 (+ 2 2)))) 


(on 2 (on 3 (+ 2 2))) 


(on 3 (+ 2 2))) 


Expression is 
evaluated 




Expression is 
evaluated 




Expression is 
evaluated 




Node 1 


Code relocates 
to node 2 


© 


Code relocates 
2 to node 3 


Node 3 


Result is 
generated 


V 


Figure 3.2: Visualization of relocation across multiple nodes in XPL 


This program will traverse the network in a similar fashion, evaluating the expression 
Node.ip(ai) on node 3. Node.ipO is a function in XPL that takes a network interface as 
an argument and returns the IP address of that interface, ai is a variable in XPLANE 
representing the network interface that the XPLANE packet arrived on. The result of the 
expression Node.ip(ai) would then be the IP address of the interface the program arrived on. 
Unfortunately, the result of this computation would stay on node 3. A more useful program 
would somehow return the result of the program to the node of origin. This is accomplished 
in XPLANE through the use of continuation passing style (CPS) transformations. 


3.2.1 Continuation Passing Style 

CPS is a programming technique in computer science in which each function or procedure 
that is called is passed a continuation. This continuation represents the remaining work of 
the computation to be performed, and is executed at the end of function instead of returning 
to the caller. This allows a program to continue execution without ever having to return to 
the point from which it was called [15]. XPLANE utilizes CPS to accumulate a continu¬ 
ation that will return the result of a computation back to the originating node. When the 
result of a program is generated, the continuation is called, carrying the result back to the 
originating node along the reverse path taken to the final node. 

In addition to the On function, XPL has a function, OnFlood, which has the syntax: 
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OnFlood( el) 


where el is any valid expression. OnFlood will evaluate the expression el on all directly 
connected neighbors, hence the name. Consider the following program: 


OnFlood( Node, ip(ai)) 


This program would flood to all directly connected neighbors which would in turn evaluate 
Node.ip(ai) and return the IP address of the arrival interface. If this code were compiled 
with the CPS transformation, the IP address of every directly connected neighbor would 
be returned to the origin. Figure 3.3 shows how the computation would take place on a 
network assuming the originating node has three directly connected neighbors. 


(node.ip ai) 

3. Expression 
is evaluated 


(onflood (node.ip ai)) 


Node 2 


2. Code relocated 
to all nodes 

< 


1. Expression 
is evaluated 


V 7 


4. Result is 
returned 


( . ' 

Node 1 


2. Code relocated 
to all nodes 

> 


2. Code relocated 
to ail nodes 


\7 


4. Result is 
returned 


m 

Node 3 


4. Result is 
returned 


Node 4 


(node.ip ai) 


3. Expression 
is evaluated 


(node.ip ai) 


3. Expression 
is evaluated 


Figure 3.3: Visualization of program utilizing OnFlood compiled with CPS transformation 


It is interesting to explore the mechanics of the above OnFlood program. By calling On- 
Flood, the current state of execution was suspended, encapsulated into an XPLANE packet, 
and broadcasted on the network. In effect, this made n copies of the original program, 
where n is the number of directly connected neighbors that are XPL-enabled. When used 
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in a recursive function, it is possible to utilize OnFlood to reach every XPL-enabled node 
in a network non-deterministically, assuming there is a subgraph connecting all such nodes. 
This is a powerful feature of XPL. Programs need not have any prior knowledge of network 
topology, and can make very few assumptions about it. A good example of a use-case for 
this feature would be locating an XPL node on the network without any prior knowledge 
of its location. Such a program would follow the general algorithm: 

Algorithm 1 Algorithm to locate a node in the XPLANE 
1: procedure FlNDNODE(«,pa/7?) > n=target node, path=list of nodes visited so far 
2: currentNode -t— Node () 

3: if currentN ode = n then 

4: return path > path contains all nodes from source to destination 

5: else if Member (currentN ode , path) = false then 

6: path -t— AppendToListfpath, currentNode) > Append the current node to path 

7: OnFlood(FindNode(n,path )) 

8: end if 

9: end procedure 


In this algorithm, the Member() function returns true if the first argument is a member of 
the second argument, and the AppendToList() function appends the first argument to the list 
provided as the second argument. An XPL implementation of this algorithm can be found 
as Listing 1 of Appendix A. The FindNode() procedure will return the path taken to the 
destination node if a path is found. It is worth noting that all paths to the destination node 
will be found due to the use of OnFlood, and therefore the originating node may receive 
multiple results. 

3.2.2 Packet Injection and Capture 

The second major feature of XPL is packet capture and injection. The XPL functions Send 
and Pcap are used to send and capture packets, respectively. The two are often used in 
conjunction to perform some kind of active measurement within the network. The syntax 
of Send is as follows: 


Send(iface, pktdesc; body) 
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where iface is a network interface, pktdesc is a packet descriptor and body is the XPL pro¬ 
gram within scope of the sent packet. The packet descriptor is used to specify parameters of 
the packet to inject such as protocol and destination host. The body will be called after the 
packet is injected. This is typically used to schedule packet capture via the Pcap function. 
Pcap has the syntax: 


Pcap(iface, filter, timeout, pktfunc) 


where iface is a network interface, filter is a tcpdump-style filter, timeout is a timeout value, 
and pktfunc is a packet-processing function. The filter will instruct Pcap to only capture 
packets that match the supplied filter. The timeout value specifies the number of seconds to 
capture on the supplied interface. The packet-processing function is responsible for parsing 
captured packets and making sense of any captured traffic. 

For example. Algorithm 2 demonstrates a ping-like program using Send and Pcap. 


Algorithm 2 Send a ping and look for reply 


1 

procedure P (pkts) 

> pkts=list of packets to process 

2 

if pkts — null then 


3 

return false 

> reply packet was not found 

4 

else 


5 

pkt Head (pkts) 

> grab the first packet from the list 

6 

if IPsrc(pkt) = ”10.0.0.1” then 


7 

return true 

> reply packet was found 

8 

else 


9 

P(Tail(pkts)) > continue processing remaining packets 

10 

end if 


11 

end if 


12 

end procedure 


13 

protoAndDest P- List(”EchoRequesf\ ”10.0.0.1” 


14 

Send( 1, protoAndDest; Pcap(\ , ”icmp",3,P)) 
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The program attempts to ping the IP address 10.0.0.1 and looks for a reply. The program 
begins by calling Send to generate and inject an "EchoRequest" packet to the IP address 
10.0.0.1 on the supplied interface, which in this case is interface 1. After the packet is 
injected. Send evaluates the supplied body which calls Pcap. Pcap schedules packet capture 
on the supplied interface (again interface 1) and filters for ICMP packets. Pcap will attempt 
to capture packets for three seconds before terminating. Any captured packets will be 
handled by the specified packet-processing function P. P takes a list of captured packets 
and looks for replies from 10.0.0.1, returning true if one is found and false otherwise. 
Listing 2 of Appendix A gives the XPL code for this algorithm. 

By combining packet capture and injection with relocation, packets can be inserted into the 
network at one node, and captured at a different node. This kind of flexibility allows for 
flexible real time measurements in a network. 
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CHAPTER 4: 

Developing the Tactical Edge Suite of Applications 


This chapter will begin to explain the rationale behind the design and development of a 
new troubleshooting toolset. The core of this toolset will consist of a library of XPLANE 
applications that specifically target tactical networks. This library of applications will be 
referred to as the tactical edge suite. Once developed, the tactical edge suite will be used 
to actively map and diagnose issues in tactical networks. This chapter will discuss the 
design philosophy of edge suite applications, as well as the testing methodology and testing 
platforms used. Individual applications will be presented and their use in tactical data 
networks will be explained. 

4.1 Design Philosophy of Application Suite 

Individually, each application is designed to serve a very specific purpose, such as finding 
nodes or discovering paths. Limiting application scope keeps the overall size of the appli¬ 
cation small and performance predictable. By combining or even chaining the output of 
multiple applications together, network operators can incrementally build on information 
discovered about network state, eventually reaching a live view of the network. Such a 
view of the network is quite different from the pre-programmed view in applications such 
as Solar Winds, which are often a representation of what the network should look like. By 
incrementally building a view of the actual network state, network operators should be able 
to make more informed decisions during troubleshooting. 

4.2 Assumptions Made for Testing Purposes 

In order to allow for testing, assumptions on the breadth of the XPLANE in a network have 
to be made. In other words, a decision always has to be made on which nodes in a network 
will be XPL-enabled. In a perfect world, all nodes would be part of the XPLANE, as this 
would allow for the most accurate view of the network from an XPLANE perspective. Lor 
the purposes of this thesis, the assumption is that all network infrastructure is XPL-enabled 
(i.e., routers and switches). In the author’s view, this is the most likely deployment strategy 
for XPLANE in future tactical networks. 
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With the design philosophy and assumptions in mind, the applications can be introduced. 
For each application, a number of points will be addressed. First, the rationale behind 
developing each application, and its relevance in tactical networks will be explained. Next, 
the application’s general algorithm will be explained and presented. Sample use cases will 
be given, and any notes on performance, behavior, and expected outputs will be given. 
Finally, the XPL source code will be listed in Appendix A. It is important to note that 
the source code listings are in uncompiled form; the code must be compiled by the CPS 
transformation to be used correctly on the XPLANE. 


4.3 Discover XPLANE Nodes Using OnFlood 

This application utilizes the XPLANE OnFlood function to discover all XPL-enabled 
nodes in the network. This application is designed to be a starting point for gathering 
network information. By first discovering the XPL-enabled nodes on the network, network 
operators can build on this information by next discovering links, IP addresses, and other 
relevant troubleshooting information. The importance of having an accurate view of the 
network topology in troubleshooting situations cannot be overstated; without an accurate 
view of the network, network operators are left at a severe disadvantage. 

4.3.1 Algorithm Description 

The application starts by making an initial call to a recursive function. This function checks 
the local XPLANE node ID against a list of already visited nodes. If the node ID is not 
found in the list, it will add the local node ID and flood to all directly connected neighbors. 
The terminating condition for the application is returning to a node already visited. When 
this happens, the list of discovered nodes is returned to the originating node. This list 
not only contains discovered nodes, but also discovered adjacencies between nodes, as 
the nodes are added in the order they were traversed. The general algorithm is shown as 
Algorithm 3. 

4.3.2 Assumptions 

The application assumes there is an XPL-enabled device within the local broadcast domain, 
which follows from the originally stated assumptions for testing. 
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Algorithm 3 Discover XPLANE nodes using OnFlood 
1: procedure DlSCOVERNODES(pa/7?) > path=list of nodes visited so far 

2 : currentNode NodeQ 

3: if Member (currentN ode } path) — true then 

4: return path > path contains all nodes visited 

5: else 

6: path AppendToList(path, currentN ode) > Append the current node to path 

7: OnFlood (DiscoverNodes(path)) 

8 : end if 

9: end procedure 


4.3.3 Additional Notes 

The application returns a list to the originating node, with the order of discovered nodes 
starting from left to right. The use of OnFlood will create multiple copies of the applica¬ 
tion, which will in turn generate many duplicate results at the originating node. In large 
networks, the number of copies of the application can grow quite large, and may affect 
network performance. In other words, this strategy of node discovery is noisy. A good 
use-case for this particular strategy would be a network with unreliable transmission paths. 
By making multiple copies of the application, the likelihood of receiving path information 
at the origin increases. For example, for just a single path of length k there will k-1 attempts 
to report the first hop of the path, k-2 attempts to report the first two hops, and so on. 

4.4 Discover XPLANE Nodes Using Depth First Search 

This application utilizes a depth-first search (DFS) approach to discover all XPF-enabled 
nodes in the network. Just as with discovery via flooding, this application is designed to 
be a starting point for gathering information on network topology, that, unlike Algorithm 
2, generates less traffic. However, it does not produce path information like Algorithm 2 
does. 

4.4.1 Algorithm Description 

This application starts by creating a list of interfaces on the current node. For each inter¬ 
face, it enumerates all directly connected neighbors via XPFANE’s Node.direct function. 
For each neighbor, the application relocates and begins the same process again. All node 
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identifiers discovered along the search are appended to a list, which is eventually returned 
to the originating node. The general algorithm is shown as Algorithm 4. 


Algorithm 4 Discover XPLANE nodes using depth-first search 
l: procedure Interfaces^, s) 

2 : if 1st = null then 

3: return 5 

4: else 

5: directNeighbors Node .direct (H ead(lst)) 

6: Interfaces(Tail(lst) ,Neighbors(directNeighbors, 5 )) 

7: end if 

8 : end procedure 

9: procedure Neighbors^, s) 

10: if 1st = null then 

11: returns 

12: else 

13: Neighbors {Tail (/ st), On(Head (/ st), Visit (s))) 

14: end if 

15: end procedure 

16: procedure Visit(s) 

17: if Member(Node() , s) = true then 

18: return s 

19: else 

20: Interfaces(Node.ifaces ( ),AppendToList(s , Nod e ())) 

21: end if 

22: end procedure 


4.4.2 Assumptions 

The applications assumes that XPLANE beaconing is turned on, or that nodes have had 
sufficient time to learn of neighbors through other means. 

4.4.3 Additional Notes 

The application returns a list of discovered node identifiers to the originating node. There is 
no implied adjacency information in the returned list. This application uses the XPLANE 
On function for relocation, and therefore does not create multiple copies of itself in the 
network. This makes the DFS-based approach less noisy than the flooding approach. It 
does however increase the chance of failure; if the XPLANE packet is dropped during any 
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relocation at runtime, the entire computation will be lost. This method of discovery should 
be avoided in highly unreliable networks. A good use-case for this application would be 
node discovery in a stable network where a less-noisy discovery process is preferred. 

4.5 Enumerate Information on a Remote Node 

This application will traverse a known path to a remote node. The path may have been 
discovered using Algorithm 3. Once at the destination node, the application will enumerate 
the IP address, subnet mask, MAC address, and directly connected neighbors for each 
interface. This information is then returned to the originating node as a nested list. 

4.5.1 Algorithm Description 

The application starts by taking a unidirectional path as input. It then begins relocating to 
nodes in the path. At each relocation, the application checks to see if the current node ID 
matches the destination node ID. If it does not, the application relocates to the next node in 
the path. If it does match, the application calls a recursive function that enumerates infor¬ 
mation on all node interfaces. Once complete, the information is returned to the originating 
node. This application has two terminating conditions. The first occurs if the application 
fails to reach the destination node ID by the specified path. The second occurs when the 
applications runs out of interfaces to enumerate on the distant node. The general algorithm 
is shown as Algorithm 5. 

4.5.2 Assumptions 

As with other protocols, a best effort attempt is given to reach the destination node. Recent 
changes in network conditions and other factors can affect the outcome of the application. 
The application will silently terminate if the destination node cannot be reached via the 
provided path. 

4.5.3 Additional Notes 

The application returns a nested list to the originating node. A use-case for this application 
would be to enumerate information on a node ID that is known to exist in the network, such 
as after running a discovery application. 
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Algorithm 5 Enumerate information on a remote node 
1: procedure TRAVERSE(pa/7r) > path=list of nodes 

2: if Head (path) = Node() then 

3: EnumNodeInfo(List(),Node.ifaces()) > List() = empty list 

4: else 

5: path -t— Tail (path) > Remove first element of path 

6: On(Head(path) 1 Traverse(path)) 

7: end if 

8 : end procedure 

9: procedure EnumNodeInfo (nodeln f oList,devs)> devs=interfaces to be enumerated 
10: if devs = null then 

11: return nodelnf oList > nodelnfoList contains info on all interfaces 

12: else 

13: d •*— Head(devs) > d=current interface 

14: devlnfo List (d , Node dp(d ), Node .mask(d ), Node .ethaddr(d ), Node .direct (d)) 

15: AppendToList (nod elnfoList , d evlnfo) 

16: EnumNodelnfo(nodeInfoListt, Tail (devs)) 

17: end if 

18: end procedure 


4.6 Re-positional Ping 

This application will traverse a known path to a distant node. Once at the destination node, 
the application will attempt to ping the IP address given as an argument. The value in this 
application comes from the ability to test connectivity from the point of view of another 
XPLANE node. Such information is essential in diagnosing routing issues, firewall issues, 
and in gaining insight to network flows. 

4.6.1 Algorithm Description 

The application starts by taking a unidirectional path as input. It then begins relocating 
to nodes in the path. At each relocation, the application checks to see if the current node 
ID matches the destination node ID. If it does not, the application relocates to the next 
node in the path. If it does match, the application calls the ping function, which will use 
XPLANE’s packet injection and capture mechanisms to test connectivity with the target 
IP address. The application will return true to the originating node on a successful ping, 
otherwise it will return false. The general algorithm is shown as Algorithm 6. 
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Algorithm 6 Re-positional ping 

1: procedure TRAVERSE(pa/7i) > path=list of nodes 

2: if Head (path) — Node() then 

3: PingUpAddr. if ace) > ipAddr=IP to receive ping, iface=interface to send on 

4: else 

5: path Tail (path) > Remove first element of path 

6: On(Head(path) ,Traverse(path)) 

7: end if 

8 : end procedure 

9: procedure Ping (destIP, dev) > dev=network adapter on host to ping from 

10: Send (dev, Listf'EchoRequest ”, destIP ); 

11: Peap(dev, 'hemp”, 3, ProcPkts(pkts, destIP ))) 

12: end procedure 

13: procedure ProcPkts (pkts, source) 

14: if pkts — null then 

15: return false 

16: else 

17: pkt — Head (pkts) > grab next packet in the list of captured packets 

18: if pkt.sourcelP = source then 

19: return true 

20: else 

21: ProcPkts (Teal (pkts), source) > continue processing list of pkts 

22: end if 

23: end if 

24: end procedure 


4.6.2 Assumptions 

Similar to Algorithm 5, a best effort attempt is given to reach the destination node. Any 
network changes could affect the outcome of the application. The application will silently 
terminate if the destination node cannot be reached via the provided path. 

4.6.3 Additional Notes 

This application is useful for testing connectivity between two distant nodes. The infor¬ 
mation returned from this application could be extremely important in scenarios such as 
troubleshooting firewall problems, access control list problems, or general connectivity is¬ 
sues between two known IP addresses that should be able to communicate. The real value 
in these scenarios is getting the network to perform actions that usually require physically 
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re-locating an administrator. By automating the test from a distance, a network operator 
need not rely on another person at the distant node. 


4.7 Testing Platform for the Tactical Edge Suite 

The approach used to test applications in the tactical edge suite is commonly referred to as 
unit testing. In unit testing, code is executed on a specific input, in a specific environment, 
and the output of that code is compared against expected results. In the case of the tactical 
edge suite, the code was each individual application, and the environment was a series of 
different networks, all configured in a way to specifically test that application. Special 
attention was paid to edge cases in order to verify the behavior of applications. 

4.7.1 CORE 

To facilitate testing, the Common Open Research Emulator (CORE) virtualization platform 
was used. CORE is a tool developed by Boeing Research and Technology and supported 
by the Naval Research Laboratory (NRL) for emulating networks [16]. Networks built 
in CORE are completely virtual; the network infrastructure and nodes all run as virtual 
machines that emulate the nodes they represent. Figure 4.1 shows an example network 
designed in CORE. 

Each router, switch, and host can be configured to run different built-in protocols and ser¬ 
vices, as seen in Figure 4.2. The interface is largely a drag-and-drop, Visio-style interface 
that has a small learning curve for experienced network designers. 

The CORE platform offers three important features for testing. The first feature is the 
ability to communicate with the virtual networks. Networks built in CORE can be bridged 
with real networks, allowing communication between physical and virtual nodes. This 
allows application development to take place on a physical machine while still allowing 
testing with a virtual network. The second feature is CORE’S ability to induce errors into 
the network. Figure 4.3 shows CORE’S link configuration menu. CORE allows users 
to artificially introduce problems such as latency, dropped packets, and duplicate packets 
into network links. This feature is crucial for emulating the performance of XPLANE 
applications in malfunctioning networks. The third feature is CORE’S built-in Topology 
Generator , which allows users to rapidly generate a network of nodes from a menu-driven 
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Figure 4.1: An example virtual network running in CORE 


interface. Users can select from a number of common topologies to include star, grid, 
cycle, and clique. CORE even has the option to generate random topologies. The CORE 
Topology Generator was key to rapid testing in multiple topologies. CORE has built-in 
Python scripting support, which allows for scripting of common tasks. CORE also comes 
with a internal debugger, and various interfaces to modify the internals of CORE itself. 
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Figure 4.2: CORE node configuration window 



Figure 4.3: CORE link configuration menu 
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CHAPTER 5: 

Developing an XPLANE server and Web-based User 

Interface 


In order to keep the state of results from tactical edge suite applications, an XPLANE server 
has been developed. The XPLANE server maintains state about a network and provides a 
basic command line interface for injecting code into the XPLANE and capturing results. 
A web-based UI has also been developed and interfaced with the XPLANE server, which 
gives network operators a detailed view of the network topology. 


5.1 XPLANE Server 

In its current state, XPLANE consists of XPL and the runtime environment [7]. XPLANE 
applications are launched via a command line program that accepts XPL-scheme and in¬ 
jects the code into the local XPLANE node. Results of computations can be "seen" by 
viewing the output from the XPLANE shim on the node where the application terminates. 
On termination, the results are not stored in the XPLANE; it is up to the user to capture, 
remember, and make sense of any information returned. This design is efficient for the 
network and XPLANE runtime environment, but makes life harder for the end user. 

Remembering and interpreting results are not the only usability issues for network opera¬ 
tors. Applications from the tactical edge suite often expect certain parameters or arguments 
to be defined, such as target node identifiers or IP addresses. Each time an application runs, 
the code needs to be hand-modified with the correct arguments. If the code is expected to 
return results from a distant part of the network, it also needs to be recompiled with the 
CPS transformation [7]. This requires users to know how to edit XPL code and compile 
programs with the CPS transformation. These barriers to usability have been overcome by 
designing an XPLANE serx’er. 

The goal of the XPLANE server is to provide a layer of abstraction between the XPLANE 
runtime system and the network operator. The server is responsible for launching appli¬ 
cations from the tactical edge suite, capturing any results, and saving the overall network 
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state. The main XPLANE server components are the command-line interface, a set of 
classes representing the tactical edge suite, and an embedded web server. The XPLANE 
server is written in the Java programming language and is therefore platform-agnostic. The 
XPLANE server requires nothing more than the current Java Runtime Environment, which 
is available on most modem platforms. All necessary support libraries are bundled with the 
server itself. 


5.1.1 Command-line Interface 

The command-line interface allows users to run edge suite applications and view results, 
display and modify network state, and push manual updates on network state via the web 
server. Communication of network state to clients will be addressed in Section 5.1.3. Fig¬ 
ure 5.1 shows a screen capture of the server’s command-line interface after both running an 
application and printing out a summary of the network state. . The command-line interface 
can be thought of as an XPLANE shell; it will infinitely loop, accepting commands and 
displaying their output until the server is explicitly stopped by the user. 


Got result: 3 1 100 

Got result: 2 3 1 100 

Got result: 421 1O0 

Got result: 3 2 1 100 

Got result: 4 3 1 100 

Got result: 3 4 2 1 100 

Got result: 3 4 2 1 100 

Got result: 2 4 3 1 100 

Got result: 3 4 2 1 100 

Got result: 2 4 3 1 100 

Got result: 4 2 3 1 100 

Got result: 2 4 3 1 100 

Got result: 4 2 3 1 100 

Got result: 4 3 2 1 100 

Got result: 4 3 2 1 100 

App discover nodes (flooding) finished. 

xpserver: run discovernodes 

App node discovery starting... 

Timeout complete. 

Got result: 100 1324 
App node discovery finished, 
xpserver: print 
Nodes: 100 1 3 2 4 

Edges: 100:1 1:100 1:3 1:2 3:1 3:2 3:4 2:4 2:3 2:1 4:2 4:3 
xpserver: | 


Figure 5.1: Screen capture of the XPLANE server's command-line interface 
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5.1.2 Edge Suite Classes 

In order to integrate the tactical edge suite in to the server, each individual application is 
"wrapped" inside a Java class. Each class consists of a string representation of the CPS 
compiled XPL code, a constructor method, and a run() method. The XPL code has been 
modified by removing all application-specific parameters and replacing them with Java 
string format specifiers. The constructor is responsible for taking arguments passed via the 
command line and formatting the XPL code string with the given arguments. The run() 
method is where the work is done. When called, run() will inject the modified XPL code 
into the local XPLANE shim. It will then begin capturing output from the XPLANE shim 
and look for results from the injected application. However, there may be no results if the 
network experiences a fault that prevents the application from completing execution. 


Dealing with Incomplete Executions 

When designing a Java wrapper class to an XPLANE application, an important design 
decision is how to cope with partial executions due to network issues. One reason for partial 
executions is the mechanism of relocation in XPLANE. Applications relocate via Ethernet 
frames, and if a frame carrying an XPLANE application is dropped during transmission, the 
computation will be lost. By design, the XPLANE does not "keep state" of computations in 
the network; it is up to each individual computation to keep any necessary state. For these 
reasons, a Java wrapper class needs to make a determination on how long it is willing to 
wait to receive results from an XPLANE application. After this timeout the wrapper class 
can either report application timeout or possibly re-run the XPLANE application. All of 
these decisions can either be left to the wrapper class or the user can be queried for what to 
do when these situations occur. 

One of the most important functions of the run() method is receiving XPLANE applica¬ 
tion results and parsing them accordingly. This operation often needs to be tailored for 
the specific XPLANE application it is wrapping, which is why a generic container for 
XPLANE applications cannot be used. Once the results have been received and parsed, the 
run() method is responsible for updating the server’s internal network state. Once the run() 
method returns, the XPLANE server may optionally push network status updates to any 
connected web clients. 
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5.1.3 Embedded Web Server 

The XPLANE server comes with an embedded Hypertext Transfer Protocol (HTTP) server 
known as Jetty. Jetty is an open source web server and Java Servlet container, that also 
supports a number of other web technologies including WebSockets [17]. Jetty is written 
in Java and maintained by the Eclipse Foundation. Jetty can run as a standalone web server 
or be embedded within a larger application. In the case of the XPLANE server, Jetty is run 
in embedded mode. When the XPLANE server is started, it launches an embedded Jetty 
instance within itself to serve web content to clients. Jetty was chosen for use with the 
XPLANE server due to it’s mature implementation of the WebSocket protocol. 

WebSockets 

The WebSocket protocol is a relatively new Internet Engineering Task Force (IETF) stan¬ 
dard. The goal of the protocol is to provide browser-based applications a way to create a 
persistent, full-duplex HTTP connection with a server [18]. The traditional client-server 
model does not work well with applications that require sporadic updates from the server 
to the client. These issues have been traditionally handled with workarounds such as Tong 
polling’ in which the client continually polls the server for information updates [18]. With 
WebSockets, clients can initiate the connection and simply wait for the server to push up¬ 
dates vice constant polling from the client side. 

Once the WebSocket connection is complete, the state of the network as determined using 
the XPLANE is pushed to the newly connected client. This state information consists of 
network nodes, edges between nodes, and metadata on nodes such as IP addresses, Ethernet 
media access control (MAC) addresses, and node type. This data is formatted in JavaScript 
Object Notation (JSON). Even though JSON is primarily used in JavaScript applications, 
it is a language-independent data format and can be interpreted by many programming 
languages. After the state of the network is initially pushed, the WebSocket connection 
will persist until the client ends the connection. Any further network state updates will 
automatically be pushed to all connected clients. 

5.2 Web-based User Interface 

In order to visualize the network state provided by the XPLANE server, a web browser- 
based UI was created. Using a web browser as the interface was chosen due to the rich en- 
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vironment of technologies that modern web browsers offer including JavaScript, WebSock- 
ets, and Hypertext Markup Language Version 5 (HTML5). A web-based UI that focuses 
on open standards such as these helps remove the need for platform-specific requirements. 
This will allow the UI to run on most modern systems without modification. 


5.2.1 Design Overview 

At its core, the UI is a JavaScript application. When the UI connects and receives the ini¬ 
tial network state, it parses the JSON data into internal array objects. It then feeds these 
arrays to a library known as Data-Driven Documents (D3). D3 is a JavaScript library for 
visualizing data sets using HTML5, Scalable Vector Graphics (SVG), and Cascading Style 
Sheets (CSS) [19]. More specifically, D3 allows data to be transformed into graphical ob¬ 
jects in the web broswer that can be interacted with and later modified via the Document 
Object Model (DOM). For the purposes of this UI, the network state is rendered with D3 
as a force-directed graph. Force-directed graphs turn networks of nodes into a physical 
simulation by assigning certain properties such as charge, friction, and gravity to nodes. 
It then allows these nodes to interact in such a way that the graph naturally finds an equi¬ 
librium. At this equilibrium, the nodes will have largely separated and un-clustered from 
each other, presenting a more readable graph. Figure 5.2 shows an example force-directed 
graph generated from XPLANE server data. By using a force-directed layout, we escape 
the issue of figuring out how to render the graph manually, which is a non-trivial problem 
to solve. 


5.2.2 Nodes and Graph Interaction 

Nodes in the graph are represented by different graphics, depending on the node type spec¬ 
ified in the JSON data. Routers display as the familiar router network icon, hosts display 
as computer monitors, and nodes of type unknown display as questions marks. The graph 
allows nodes to be rearranged using the mouse; all other nodes in the graph will respond to 
changes in the force layout and react accordingly. This can be useful if the simulation failed 
to find a suitable equilibrium. Nodes will respond to the mouse-over event by displaying a 
pop-up menu. The pop-up menu will display all information available on a node. Figure 
5.3 shows an example pop-up for a test network. 
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Figure 5.2: An example force-directed graph generated from XPLANE server data 
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Figure 5.3: Ul pop-up window on node mouse-over event 


5.2.3 Server Updates 

Once the UI has successfully downloaded, parsed, and rendered the JSON network data, the 
WebSocket connection created with the XPLANE server will remain open. Any changes in 
network state will automatically be pushed from the server to all connected clients. When 
the UI receives an update, it will parse the new JSON data and compare the new data to 
the old network data. New nodes and connections will be added to the graph, and deleted 
nodes and connections will be removed. This method of updating the graph prevents D3 
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from completely re-rendering the force-directed layout from scratch on every update, which 
would make the graph unstable. Nodes and connections that persisted between updates will 
stay relatively close to their original positions and will adjust only for new or deleted nodes. 

5.2.4 Anomaly Detection 

In addition to rendering the network visually, the UI will also attempt to detect anomalies 
in the network. Anomaly detection occurs outside of the XPLANE by processing the raw 
JSON network data in the UI and looking for known network anomalies. At this time, 
the only anomaly successfully detected is duplicate IP addresses. Detected anomalies are 
displayed in a table within the UI. Figure 5.4 shows a screen capture with two detected 
network anomalies. Anomaly detection is run every time a network update is received, and 
the detected anomalies table is updated accordingly. 



Figure 5.4: Screen capture of detected anomalies table 
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CHAPTER 6: 
XPLANE Case Study 


In order to compare the utility of the XPLANE toolset presented in Chapters 4 and 5 against 
traditional tools, a case study was conducted. For this study, a battalion-sized tactical data 
network was constructed in a laboratory setting. The case study is focused around the 
scenario of troubleshooting a geographically dispersed network from a central location 
much l ik e the author’s experience in Afghanistan. First the network used for the case study 
will be presented. Next an attempt to troubleshoot the network using traditional tools and 
protocols will be presented. This will be followed by an attempt to troubleshoot the same 
scenario using the XPFANE toolset presented in this thesis. Finally the results of both 
troubleshooting attempts will be compared. 

6.1 Network Layout 

The network consists of four notional company positions: Alpha, Bravo, Charlie, and 
Headquarters. Each company position consists of a router, a switch, and two worksta¬ 
tions. The number of workstations was kept small for simplicity. The company positions 
are connected through a number of means. Headquarters is directly connected to all three 
companies. Alpha and Charlie company are notionally located in the same geographic area 
and share a redundant link between each other. Bravo company is geographically isolated 
from the other companies. All companies are notionally operating at a significant distance 
from Headquarters company; travelling to other company positions for troubleshooting is 
a last resort. Each company is responsible for a class C subnet. Static routing was utilized 
due to the static nature of company positions. Figure 6.1 shows a diagram of the network 
architecture. 

Once the network was functioning correctly, errors were introduced at the logical layer to 
emulate a malfunctioning tactical network. The errors introduced were common misconfig- 
urations inspired from the author’s operational experience. The network has been logically 
broken in three ways: 

1. The routing table on Alpha company’s router has been accidentally deleted during a 
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Figure 6.1: Network diagram for case study before errors are introduced 


hypothetical troubleshooting session. 

2. Bravo company’s router has an incorrect subnet mask specified for the link to Head¬ 
quarters company. 

3. Charlie company’s router has an incorrect (and duplicate) IP address on the interface 
connected to Headquarters company. 

A complete description of the network configuration and induced errors can be found in 
Table B.l in Appendix B. No virtualization was employed for the case study; all nodes are 
physical. The tactical transmission systems that would connect company positions in a real 
world scenario are notional. Ethernet cables were used to connect company positions in 
the laboratory setting. For the purposes of troubleshooting we will assume that the notional 
transmission systems have been verified as working correctly. Each router is a Linux- 
based Soekris router. The XPLANE shim requires a Linux-based environment at this time, 
which Soekris routers provide. The XPLANE shim is running on all four routers as well as 
the troubleshooting workstation in the Headquarters subnet. The node identifiers for each 
XPLANE node are displayed in Figure 6.1. The XPLANE shims have beaconing enabled. 
All troubleshooting will take place from the XPL-enabled node within the Headquarters 
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network (XPLANE node ID 151). 


6.2 Troubleshooting with Traditional Tools 

The first troubleshooting attempt was made with the tools and protocols described in Sec¬ 
tion 2.3. For each tool, a description of the way it was employed as well as the results of 
all tests performed will be explained. 

6.2.1 Ping 

The first step in network troubleshooting is to test logical layer connectivity between hosts. 
In this case, the connectivity between Headquarters and all three companies is in question. 
The first tool often employed for this test is the ping program or some version of it. For 
this case study, we will utilize the program Nmap to automate a network-wide ping scan on 
our behalf. Nmap will send a ping packet to every host in the specified networks and report 
back any replies. Even though we have a good idea of which hosts to ping from the network 
documentation, we will tell Nmap to scan the four class C subnets in use with the following 
command: nmap -sP 10.0.1.0/24 10.0.2.0/24 10.0.3.0/24 10.0.4.0/24 10.0.10.0/24. Nmap 
produced the following output: 

Nmap scan report for 10.0.1.1 
Host is up (0.00081s latency). 

Nmap scan report for 10.0.1.51 
Host is up (0.00028s latency). 

Nmap scan report for 10.0.10.1 
Host is up (0.00044s latency). 

Nmap scan report for 10.0.10.5 
Host is up (0.00098s latency). 

Nmap scan report for 10.0.10.9 
Host is up (0.0015s latency). 

Nmap done: 1280 IP addresses (5 hosts up) scanned in 20.88 seconds 

Fooking closely at the live IP addresses we quickly notice that the only replies received 
were from our own workstation and the interfaces on the Headquarters router. This adds 
evidence to the possibility that logical layer connectivity is down with all three companies 


43 



for some reason. It really only confirms that we cannot send ICMP traffic to the company 
networks; it says nothing for other logical layer protocols. The next step would be to verify 
all settings on the Headquarters router, which would produce no answers since it correctly 
configured. In order to confirm that the issue affects the entire logical layer, we can use 
Nmap again to perform a TCP-based ping. This will make Nmap attempt a TCP connection 
to a specific port to see if any response is received at all. The following command was used: 
nmap -PS22 10.0.2-4.1. This tells Nmap to send a TCP connection request to port 22 of 
each company router and look for any reply. In our case, port 22 of the routers is running 
secure shell (SSH) and therefore should reply. Nmap produced the following output: 

Nmap done: 3 IP addresses (0 hosts up) scanned in 2.07 seconds 

It is now clear that the logical layer is broken in some way. Unfortunately we have ex¬ 
hausted the utility of connectivity testing tools such as ping and must move on. 

6.2.2 Traceroute 

Traceroute is used in troubleshooting to discover logical paths between two nodes. Tracer¬ 
oute will show each hop along the path, and more importantly, the last successful hop. If 
the last hop is not the target node, then we conclude that the source of the problem lies 
somewhere close to the last successful hop along the path. In our case, traceroute will 
be of little value. We have already shown that the logical layer is malfunctioning, which 
traceroute relies on. On top of that, we are able to validate our own router’s configuration 
which appears to be error-free. Regardless, it is never a bad idea to try all available tools. 
Upon running traceroute on 10.0.2.1,10.0.3.1, and 10.0.4.1, the tool confirmed the last hop 
before failure is the Headquarters router. This at least confirms that traffic is making it from 
our workstation to the Headquarters router. Beyond that, no other information is provided 
for our troubleshooting efforts. 

6.2.3 SNMP 

SNMP can often be used in troubleshooting situations as long as network devices support 
the protocol. In our case it makes no difference. SNMP is a UDP-based protocol that lives 
at the logical layer of the network. Until we can fix the logical layer, tools that exist at this 
layer will be of limited value. 
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6.2.4 Link Layer Discovery Protocols 

Protocols such as Link Layer Discovery Protocol (LLDP) cannot be applied to help diag¬ 
nose problems in routed networks. Our case study network is segmented into different class 
C subnets for each location. Link layer discovery protocols are limited to the local subnet 
by definition. They cannot help in troubleshooting problems across subnet boundaries. 

6.2.5 Results with Traditional Tools 

We were able to gather evidence that there are logical layer connectivity issues to all three 
companies. We are able to verify that the Headquarters’ router configuration is correct. 
Combined with the previous assumption that transmission systems are working correctly, 
we conclude that there are logical configuration issues on all three company routers. We 
have no idea what the issues are and therefore need an experienced network operator to 
troubleshoot each router configuration. If we do not have a seasoned network operator at 
each location we would have to physically send one. 

6.3 Troubleshooting with XPLANE 

The second troubleshooting attempt was made with the tools presented in Chapters 4 and 
5. As a reminder to the reader we are XPLANE node ID 151 and the XPLANE shim is 
running on all routers. We begin by starting the XPLANE server and connecting the web- 
based UI. At this point the only thing we know about in the XPLANE is ourselves. Figure 
6.2 shows our current view of the network. 





w 

151 


Figure 6.2: Initial view of the XPLANE during troubleshooting 


6.3.1 Node Discovery with Depth-first Search 

We start by attempting to discover nodes in the network using the depth-first search discov¬ 
ery application discussed in Section 4.4. Figure 6.3 shows our view of the network after 
depth-first search discovery. 
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Figure 6.3: View of the XPLANE after node discovery 


We discovered four additional nodes numbered one through four which aligns with our 
network documentation. At this point we would like to query each node for information but 
we do not know how each node is connected. We have an additional discovery application 
in our toolbox that we can use. 

6.3.2 Path Discovery with OnFlood Search 

The discovery application described in Section 4.3 gives us both node and path information, 
so we will use that next. Figure 6.4 shows the network view after path discovery. 



Figure 6.4: View of the XPLANE after path discovery 
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6.3.3 Router and Host Enumeration using XPLANE 

The network topology now looks similar to what was expected. We continue by using the 
application described in Section 4.5 to enumerate information from each node. Figure 6.5 
shows the network after enumerating nodes one and two. 


XPL-enabled yes 
Node ID 2 
Interface '2': 10.0.2.1 

255.255.255.0 
00:00:24:CE:5E:04 
Interface '3': 10.0.10.2 

255.255.255.252 

00:00:24:CE:5E:05 151 



Figure 6.5: View of the XPLANE after enumerating first two nodes 


The information pop-up shows all enumerated information for node two. Node two is 
clearly Alpha company’s router. Now that we have some basic information on its live con¬ 
figuration we can begin to troubleshoot. Comparing the live configuration to the network 
documentation yields no discrepancies. Yet for some reason we still cannot communicate 
with Alpha company which suggests the issue lies deeper. Unfortunately XPLANE does 
not currently provide any additional information on Alpha’s router. We can however run 
XPL applications on Alpha’s router, which is what we will do next. We will attempt to 
discover the two hosts on Alpha’s subnet using the re-positional ping program described in 
Section 4.6. The program will attempt to ping the IP addresses of the hosts from Alpha’s 
router and report back the results. Figure 6.6 shows the results of our ping attempts. 
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Figure 6.6: View of Alpha's network after discovering hosts 


We have discovered two additional nodes that have been assigned node identifiers 1000 
and 1001. Since XPLANE is not running on edge devices, they have no XPLANE node 
identifier. The XPLANE server makes up for this by generating a unique node identifier 
for nodes that lie outside the XPLANE. The current numbering scheme starts at 1000 and 
increases by one for each new node found. The number can be anything as long as it is 
unique. The information pop-up shows all enumerated information for node 1000 which 
really only consists of the host’s IP address. The identified information shows that these 
are in fact the two hosts we expected to find in Alpha’s network. We will continue with 
the same process for both Bravo and Charlie’s subnets. Figure 6.7 shows the graph view of 
node four as well as the anomaly table after enumerating node four. 

Once enumerated, it becomes clear that Charlie’s router has a wrong (and duplicate) IP ad¬ 
dress on its connection with Headquarters. The anomaly was detected by the web-based UI 
and pointed out in the anomaly table. Charlie’s connection to Headquarters should have an 
IP address of 10.0.10.10', the last octet is incorrect. We next enumerate the hosts in Char¬ 
lie’s subnet in the same manner as Alpha’s. Figure 6.8 shows the results of enumeration. 
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• XPL-enabled yes 
I Node ID: 4 

/ Interface '2': 10.0.4.1 
/ 255.255.255.0 

/ 00:00:24:CE:5E:00 

/ Interface '3': 10.0.10.9 
. / 255.255.255.252 

00:00:24:CE:5E:01 
^1 Interface '4 1 : 10.0.10.14 

255.255.255.252 

00:00:24:CE:5E:02 


* 
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Time 

Nodes Involved 
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05/01/2014 09:33 AM 

1.4 


Figure 6.7: View of XPLANE after enumerating Charlie's router 



Figure 6.8: View of Charlie's network after dicovering hosts 


We have successfully discovered the two hosts in Charlie’s subnet, which are now labeled 
as nodes 1002 and 1003. As with the hosts in Alpha’s network, we will be unable to 
enumerate anything besides the hosts’ IP address. We then turn to enumeration of Bravo’s 
subnet. Figure 6.9 shows the result of enumerating Bravo’s router. 

There are no new entries in the anomaly table. At this time the web-based UI can only 
detect duplicate IP addresses, so we will need to look deeper to find problems with Bravo’s 
router configuration. 
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Figure 6.9: View of XPLANE after enumerating Bravo's router 


Upon inspection of the information provided by XPLANE, we notice an incorrect subnet 
mask on Bravo’s connection to Headquarters. The subnet mask is 255.255.255.254 which is 
incorrect. The last octet should be 252. This could explain the lack of logical connectivity 
to Bravo company. We complete the enumeration by discovering the hosts in Bravo’s 
subnet. Figure 6.10 shows the result of enumerating Bravo’s subnet and gives our final 
view of the network as discovered by XPLANE-based tools. 

6.4 Comparison of Troubleshooting Attempts 

Troubleshooting with traditional tools in this case yielded little information. Troubleshoot¬ 
ing with XPLANE-based tools yielded the entire network topology. Our XPLANE-based 
tools also yielded the configuration of all network interfaces (with the exception of routing) 
on XPL-enabled nodes. With the first attempt, we were able to rule out the Headquarters 
router configuration as a source of the problem but were unable to go beyond that. Without 
remote access to company routers we were unable to continue without relying on the phys¬ 
ical presence of a trained network operator at each location. With the information provided 
by the second troubleshooting attempt we were able to discover an incorrect and duplicate 
IP address on Charlie’s router as well as an incorrect subnet mask on Bravo’s router. Fix¬ 
ing these two issues remotely would fix the logical layer connection between Headquarters, 
Charlie, and Bravo. We were unable to diagnose the connection problems with Alpha. We 
could attempt to use the redundant link between Charlie and Alpha company once Charlie’s 
connection to Headquarters is fixed, but we would still find that we can not connect. We 
have reached the limit of what this application suite can achieve. How it might be extended 


50 





with another application to diagnose Alpha’s connection problem is addressed in Section 
7.2.1. 



Figure 6.10: View of the entire network as discovered by XPLANE 
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CHAPTER 7: 

Future Work and Conclusions 


The work presented in this thesis only begins to scratch the surface of what is possible with 
an XPLANE-like system. Both XPLANE and the XPLANE-based tools developed in this 
thesis could benefit from additional capabilities and work. We will first address XPLANE 
and the future capabilities we would like to see. Next we will address the XPLANE-based 
tools presented in Chapters 4 and 5 in the same manner. Finally we will present conclusions 
to this research. 

7.1 Future Work on XPLANE 

XPLANE is an ongoing project. Development efforts have focused on the design of XPL 
and ensuring the core capabilities of the system function correctly [7]. We will now address 
capabilities that would increase the utility of XPLANE in tactical networks. 

7.1.1 Security 

The designers of XPLANE always considered security to be future work [7]. A security ar¬ 
chitecture is driven by a threat model and without such a model, arguing a system is secure 
amounts to secure by definition. Different threat models impose different sets of demands 
on the system. We will define a threat model based on the types of networks discussed in 
this thesis (i.e., tactical networks) and explore security concerns and mitigations. 

Threat Model 

The networks considered in this thesis are tactical networks in a field environment. The 
most likely points of entry to the XPLANE in such networks would be through a device on 
the local network, open switch ports, or the web-based UI to the XPLANE server. These 
are the points of access we will consider for our threat model. 

Consider a network that is XPLANE enabled but has no mechanism for access control to 
XPLANE at the Ethernet frame level. Any host on the network could inject arbitrary XPL 
code into the XPLANE. If the code is poorly written it could have unintended errors such 
as failure to terminate. Consider the following program: 
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Algorithm 7 A non-terminating program using OnFlood 
1: procedure Foo 
2: OnFlood{Foo {)) 

3: end procedure 

4: Foo() 


Algorithm 7 is an example of a program that fails to terminate. The program additionally 
uses OnFlood for relocation and as a side effect generates many copies of itself. The 
combination of no terminating condition with flooding will cause this program to eventually 
consume all available network bandwidth. This example illustrates the need for access 
control to XPLANE at the frame layer of the network to protect against both naive users 
and malicious insiders. Naive users can be considered anybody on the network who has the 
capability to either accidentally inject code or purposefully inject untested code. Malicious 
insiders can be considered anybody who wishes to use XPLANE in unintended ways such 
as to disrupt network performance. Hence access control at the frame level is needed. 

Access Control at the Frame Level 

By implementing access control at the frame level we can restrict code injection to users 
of the XPLANE server. XPLANE has experimental support for access control at the frame 
level using message authentication codes (MACs). MACs are a cryptographic technique 
used to authenticate messages between a sender and receiver [20]. MACs are typically 
functions with two arguments that take the form: 

MAC(K,m ) 

where K is a fixed size cryptographic key and m is the message to be authenticated. The 
output of the MAC function is a fixed size value that is unique to the combination of K and 
m. When the sender wishes to authenticate message m to the receiver, the sender will send 
m as well as MAC(K,m). It is assumed that the sender and receiver have already agreed on 
the secret key K through other means. Upon receipt of the message, the receiver calculates 
MAC(K,m) and compares it to the MAC value sent with the message. If it matches, the 
receiver concludes the message has not been tampered with. If it does not match, the 
receiver discards the message. As long as an attacker does not know the secret key K, they 
cannot change the contents of authenticated messages. 
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The XPLANE currently has support for HMAC-SHA-1 [21]. HMAC is a keyed hash used 
for message authentication. It can be used with any one of several cryptographic hash func¬ 
tions. In the XPLANE, it is used with the Secure Hash Algorithm (SHA-1) hash function. 
The HMAC-SHA-1 function takes as input a secret key K and a message m where m con¬ 
sists of the XPLANE header and payload. When HMAC is enabled, XPLANE will attempt 
to authenticate XPLANE packets before running the marshaled code contained within the 
packet. If the packet does not authenticate, the XPLANE shim discards the packet. 

This is a good start for access control at the frame level. The next piece would be protec¬ 
tion from replay attacks. In a replay attack, the attacker takes advantage of the fact that 
both the message and MAC can be eavesdropped, recorded, and later resent to the original 
receiver of the message. The notion of unique message numbers is often used to prevent 
replay attacks in protocols that use MACs for authentication [20]. Implementing a message 
numbering system in XPLANE could prevent replay attacks. 

It is worth noting again that MACs only provide authentication. MACs by themselves do 
no provide any form of confidentiality. A malicious insider could eavesdrop on the results 
of XPLANE applications and leam information about the network they were not intended 
to have. In a different threat model this could be an issue. 

Use of HMAC addresses the threat of unauthorized users injecting code into the XPLANE, 
providing these users can be prevented from getting the HMAC key. If this can be guaran¬ 
teed somehow then access to the XPLANE reduces to accessing it through the XPLANE 
server. Therefore, steps must be taken to limit access to the XPLANE server to authorized 
users only. 

Access Control at the XPLANE Server 

The mechanisms for access control at the XPLANE server level are aimed at limiting 
user access. The XPLANE server should be augmented with user authentication mech¬ 
anisms. Authentication could be integrated with pre-established user credentials such as 
those stored on an existing Active Directory server. Users would authenticate via web 
forms before gaining access to the live view of the network. Once authenticated, users 
would be restricted to a set of verified applications such as the tactical edge suite. These 
applications would be guaranteed to perform in a predictable manner and always terminate. 
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Restricting users to a set of verified applications solves the problem of dealing with non¬ 
terminating code; however, it could introduce a new problem. By centralizing control of 
both XPLANE and the XPLANE server we have given up a key capability of XPLANE. We 
are now relying on a logical layer protocol (i.e., HTTP) to communicate with the XPLANE 
server and inject code into the XPLANE. Since XPLANE is meant to be used in situations 
where the logical layer is malfunctioning, we run the risk of cutting off XPLANE access to 
users that have no logical path to the XPLANE server. This is one reason that centralizing 
the XPLANE server might not be the best solution. There are likely others. If the set of 
applications available to users cannot be controlled then administrators need another option 
for dealing with non-terminating code. 

Time to Live Values 

One possible mechanism is to limit the life of an XPLANE program in the network via 
a TTL value. The idea of a TTL is not a new one. The IP protocol implements TTL 
values as an integer that decrements at each hop. When the TTL reaches zero, the packet 
is discarded. This prevents IP packets from traversing the network indefinitely [1], The 
same mechanism could be applied to XPLANE packets. Upon injection into the network, 
the TTL of a program’s XPLANE packet would be initialized to a default value. With 
each relocation in the network the TTL would be decremented until it reaches zero. Any 
new copies of the program generated with constructs such as OnFlood would inherit the 
current TTL value of the program. This ensures that programs cannot circumvent the TTL 
simply by producing new copies of themselves. While this neatly takes care of the non¬ 
termination problem, it introduces new complexities for XPL application writers. Consider 
algorithm 3, which discovers network topology using OnFlood. In large networks, this 
algorithm could possibly exhaust the TTL value of the program, forcing early termination. 
In the program’s current form, the user would have no idea whether the program terminated 
naturally or prematurely. This might leave parts of the network undiscovered if there is no 
way to alert the user to early termination. One possible solution would be to expose the TTL 
value of XPL programs to the program itself. This would allow XPL application writers to 
query the TTL value before or after relocation. This information could be used to make a 
determination on what to do when the program’s TTL begins to approach zero. Possible 
actions would be to alert the user to the forced early termination, and possibly return partial 
results of the computation. If the program returns results, it would additionally have to keep 
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track of the number of hops that would be required to return back to the originating node, 
as these hops will count against the TTL. 


Key Management 

HMAC cryptographic key management has many issues. They include generation, rollover 
induced by network speed, expiration due to policy or compromise, and distribution. Stan¬ 
dards such as the Federal Information Processing Standard (FIPS) 140-2 come into play, 
and the generation, destruction, and accountability of keying material would likely fall un¬ 
der the Electronic Key Management System (EKMS). These issues are beyond the scope 
of this thesis. 

7.1.2 Injection of TCP and UDP Packets 

XPLANE currently only supports injection of ICMP packets. Adding support for both TCP 
and UDP would allow for a broader range of measurements. XPLANE applications could 
be written to troubleshoot specific ports and application layer protocols by mimicking their 
behavior at the logical layer and observing network behavior. 

7.1.3 Querying a Node’s Routing Tables 

XPLANE has no built-in support to view a node’s routing tables. The case study presented 
in Chapter 6 is a good example of a troubleshooting situation that could have benefited 
from such support. The ability to query a distant node’s routing tables would open the door 
to detecting anomalies in routing. 

7.1.4 Querying a Node’s Management Information Base 

If XPLANE is able to find a physical path to an XPLANE-enabled node it could attempt 
to query the node’s MIB for relevant troubleshooting information. This information could 
then be transported back to the originating node via the reverse path taken. Such informa¬ 
tion could be useful during troubleshooting sessions where SNMP is running on nodes but 
unavailable due to logical layer issues. 

7.2 Future Work on XPLANE-based Tools 

The tools presented in Chapters 4 and 5 are largely proofs of concept to demonstrate the 
possibilities with XPLANE-based tools. There is plenty of room for improvement in all 
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three major components. 


7.2.1 Improving the Tactical Edge Suite 

The tactical edge suite only consists of four applications at this time. These applications 
are largely focused on network discovery. The suite is lacking in applications that make 
better use of packet capture and injection to discover interesting things about the network. 

One example of this would be enumerating all logical paths between a source and destina¬ 
tion. This could be broken down into an application that attempts to discover all physical 
paths between a source and destination and an application that tests a given physical path 
for a corresponding logical path. 

Chapter 6 demonstrated the need for applications that detect faulty paths. In the presented 
case study, the troubleshooting workstation was unable to detect anomalies in Alpha Com¬ 
pany’s router with the current tactical edge suite. An application to detect faulty reverse 
paths could be made. In the case of Alpha’s router, the application would schedule a packet 
capture on the interface to Headquarters (ethl) as well as the interface to Alpha’s subnet 
(ethO). Next it would initiate an echo request from Headquarters to a workstation in Alpha’s 
subnet. If an echo reply is seen at ethO and not ethl, there is a faulty reverse path. 

Another useful addition to the tactical edge suite would be an application that relocates 
to routers and enumerates all hosts (including non XPLANE-enabled hosts) on a given 
interface. This task is currently performed manually by the XPLANE server operator and 
is unsuitable for enumeration of large subnets. 


7.2.2 Improving the XPLANE Server 

At this time the XPLANE server can only launch XPLANE applications from the 
command-line interface. There is no support for launching applications on behalf of a 
web-based client. Additionally the XPLANE server has been built around the concept of a 
single user that runs applications in sequential manner. Adding support to take commands 
from web-based clients will necessitate either an application queueing system or a way to 
block requests from clients (as well as the command line) while an application is running. 
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7.2.3 Improving the Web-based UI 

The web-based UI has no support for launching XPLANE applications on the XPLANE 
server. This support would require the UI to be aware of the applications available on the 
XPLANE server as well as how to specify the necessary parameters to launch the applica¬ 
tion. Ideally users would specify parameters using the graph by selecting elements in the 
graph such as nodes and adjacencies. 

Anomaly detection is currently performed within the UI. As XPLANE gains capabilities 
(e.g., the ability to query routing tables) the UI will need to be augmented to discover 
anomalies in the provided data. Anomalies are currently reported to the user via the 
anomaly table within the UI. The anomaly table could be improved to provide additional 
contextual information on each anomaly to ease in troubleshooting. For instance, clicking 
on an entry in the anomaly table could highlight nodes in the graph that are involved with 
the detected anomaly. 

The UI is also lacking basic navigation features within the graph such as pan and zoom. 
Such features will be necessary to navigate larger networks using the force-directed graph. 


7.3 Conclusions 

The problem of troubleshooting a geographically-dispersed network in a combat zone is 
a hard one. Tactical networks will only get more complex from here and further exac¬ 
erbate the issue. Units will always plan to have trained people in the right place at the 
right time, but it rarely works that way. Even after our recent experiences in Iraq and 
Afghanistan we have no good solution to this problem. The Marine Corps Center for 
Lessons Learned (MCCLL) web site is host to recent after action reports from units that 
cite the same communications issues experienced in this thesis. How do we cope with log¬ 
ically broken data systems when no one can reach them? We simply do not have the tools 
to answer this question. The Marine Corps and other services need new options. 

This thesis scratched the surface of that problem and demonstrated there are options avail¬ 
able through existing technology. The addition of new protocols and networking tech¬ 
nologies (i.e., active networking) to tactical networks opens up new network management 
possibilities. We can begin to rely on live network measurements instead of broken network 
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documentation that may or may not reflect reality. We do not have to rely on a Marine’s 
physical presence at the distant end in order to perform the most mundane of tasks. We can 
query the network in ways not possible before. We can continue to grow these capabilities 
if we begin to embrace open networking platforms that will allow for rapid adoption of 
future networking technologies. 

The problems discussed in this thesis are not exclusive to the fighting experienced in Iraq 
and Afghanistan. These issues will exist in future Marine Corps networks. The landing 
forces of Marine Expeditionary Units will experience these issues when they cannot com¬ 
municate back to the command element due to a typo in a router. Mobile ad-hoc networks 
will become unmanageable when the network operations center has no way to gain a live 
view of the network. The networks of tomorrow will suffer from the problems of today if 
we do not address our core networking shortfalls. 
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APPENDIX A: 
Code Listings 


Listing A.l: Use OnFlood to find a node ID '2' in the XPLANE 
(letrec ((findnode 
(lambda (n path) 

(if (equal? n node) 

(cons node path) 

(if (not (member node path)) 

(let ((n2 node)) 

(onflood (findnode n (cons n2 path))))))))) 

(findnode 2 ' () )) 


Listing A.2: Send an 'ping' on interface 1 to 10.0.0.1 and look for reply 
(letrecp ((p (lambda (pkts) 

(if (null? pkts) 

#f 

(if (equal? "10.0.0.1" (ip.src (car pkts))) 

#t 

(p ( cdr pkts))))))) 

(send 1 ' ("EchoRequest" "10.0.0.1") 

(pcap 1 "icmp" 3 p))) 


Listing A.3: Discover all XPLANE nodes using OnFlood 

(letrec ((f 

(lambda (path) 

(if (member node path) 
path 

(let ((n node)) 

(onflood (f (cons n path)))))))) 

(f '())) 


Listing A.4: Discover all XPLANE nodes using depth-first search 
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(letrec ((visit 
(lambda (set) 

(letrec ((neighbors 
(lambda (list si) 

(if ( null? list) 
si 

(neighbors (cdr list) (on (car list) (visit si))))))) 
(letrec ((if aces 
(lambda (1 s2) 

(if (null? 1) 
si 

(ifaces (cdr 1) (neighbors (node.direct (car 1)) 

s2))) ) ) ) 

(if (member node set) 
set 

(ifaces (node.if aces) (cons node set)))))))) 

(visit 1 ())) 

Listing A.5: Enumerate information on a remote node 
; the variable path_to_dest must be defined and be a valid 
; path to the destination node ID 
(letrec ((enuminfo 
(lambda (1st devs) 

(if (null? devs) 

1st 

(let ((d (car devs))) 

(enuminfo (append 1st (list (list d (node.ip d) 

(node.mask d) (node.ethaddr d) (node.direct d)))) 

( cdr devs))))))) 

(letrec ((traverse 
(lambda (path) 

(if (and (equal? (car path) node) (null? (cdr path))) 
(enuminfo (list node.type) (node.if aces)) 

(let ((newpath (cdr path))) 

(on (car newpath) (traverse newpath))))))) 
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(traverse path_to_dest))) 


Listing A.6: Re-positional Ping 

; the variables dest_ip, dest_interface, and path_to_dest 
; must be defined 
( letrecp ((proc 

(lambda (pkts src) 

(if (null? pkts) 

#f 

(if (equal? src (ip.src (car pkts))) 

#t 

(proc (cdr pkts) src)))))) 

(letrec ((ping 

(lambda (dest dev) 

(send dev (list "EchoRequest" dest) 

(pcap dev "icmp" 3 (lambda (p) (proc p dest))))))) 
(letrec ((traverse 
(lambda (path) 

(if (and (equal? (car path) node) (null? (cdr 
path))) 

(ping dest_ip dest_interface) 

(let ((newpath (cdr path))) 

(on (car newpath) (traverse newpath))))))) 
(traverse path_to_dest)))) 
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APPENDIX B: 

Case Study Network Configuration 


Location 

Type 

Interface - IP/Mask 

Routing Table 

HQ 

Router 

ethO- 10.0.1.1/24 
ethl - 10.0.10.1/30 
eth2- 10.0.10.9/30 
eth3 - 10.0.10.5/30 

10.0.1.0/24 dev ethO 

10.0.2.0/24 via 10.0.10.2 dev ethl 
10.0.3.0/24 via 10.0.10.6 dev eth3 
10.0.4.0/24 via 10.0.10.10 dev eth2 

HQ 

Workstation 

ethO - 10.0.1.50/24 

default via 10.0.1.1 dev ethO 

HQ 

Workstation 

ethO - 10.0.1.51/24 

default via 10.0.1.1 dev ethO 

Alpha 

Router 

ethO - 10.0.2.1/24 
ethl - 10.0.10.2/30 
eth2- 10.0.10.13/30 

10.0.2.0/24 dev ethO 

10.0.1.0/24 via 10.0.10.1 dev ethl 1 
10.0.3.0/24 via 10.0.10.1 dev ethl 
10.0.4.0/24 via 10.0.10.14 dev eth2 

Alpha 

Workstation 

ethO - 10.0.2.50/24 

default via 10.0.2.1 dev ethO 

Alpha 

Workstation 

ethO - 10.0.2.51/24 

default via 10.0.2.1 dev ethO 

Bravo 

Router 

ethO - 10.0.3.1/24 
ethl - 10.0.10.6/31 2 

10.0.3.0/24 dev ethO 

10.0.1.0/24 via 10.0.10.5 dev ethl 
10.0.2.0/24 via 10.0.10.5 dev ethl 
10.0.4.0/24 via 10.0.10.5 dev ethl 

Bravo 

Workstation 

ethO - 10.0.3.50/24 

default via 10.0.3.1 dev ethO 

Bravo 

Workstation 

ethO - 10.0.3.51/24 

default via 10.0.3.1 dev ethO 

Charlie 

Router 

ethO - 10.0.4.1/24 
ethl - 10.0.10.9/30 3 
eth2 - 10.0.10.14/30 

10.0.4.0/24 dev ethO 

10.0.1.0/24 via 10.0.10.9 dev ethl 
10.0.2.0/24 via 10.0.10.13 dev eth2 
10.0.3.0/24 via 10.0.10.9 dev ethl 

Charlie 

Workstation 

ethO - 10.0.4.50/24 

default via 10.0.4.1 dev ethO 

Charlie 

Workstation 

ethO - 10.0.4.51/24 

default via 10.0.4.1 dev ethO 


Table B.l: Case study network configuration 


'These required routing entries have been accidentally deleted on the router during troubleshooting. 

2 The subnet mask is incorrect. It should be /30. 

3 The IP address is incorrect. The last octet should be 10. 
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